SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Software reliability testing</title>

</head>

<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Software reliability testing</span></span>

</h1>

</header>

<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr"><p><b>Software reliability testing</b> is a field of software-testing that relates to testing a software's ability to function, given environmental conditions, for a particular amount of time. Software reliability testing helps discover many problems in the <a href="Software_design" title="Software design">software design</a> and functionality.

</p>

<div class="mw-heading mw-heading2"><h2 id="Overview">Overview</h2></div>

<p>Software reliability is the probability that software will work properly in a specified environment and for a given amount of time. Using the following formula, the probability of failure is calculated by testing a sample of all available input states.

Mean Time Between Failure(MTBF)=Mean Time To Failure(MTTF)+ Mean Time To Repair(MTTR)

</p>

<dl><dd>Probability = Number of failing cases / Total number of cases under consideration</dd></dl>

<p>The set of all possible input states is called the input space. To find reliability of software, we need to find output space from given input space and software.<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup>

</p><p>For reliability testing, data is gathered from various stages of development, such as the design and operating stages. The tests are limited due to restrictions such as cost and time restrictions. Statistical samples are obtained from the software products to test for the reliability of the software. Once sufficient data or information is gathered, statistical studies are done. Time constraints are handled by applying fixed dates or deadlines for the tests to be performed. After this phase, design of the software is stopped and the actual implementation phase starts. As there are restrictions on costs and time, the data is gathered carefully so that each data has some purpose and gets its expected precision.<sup id="cite_ref-Lewis_2-0" class="reference"><a href="#cite_note-Lewis-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup>

To achieve the satisfactory results from reliability testing one must take care of some reliability characteristics.

For example, Mean Time to Failure (MTTF)<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup> is measured in terms of three factors:

</p>

<ol><li>operating time,</li>

<li>number of on off cycles,</li>

<li>and calendar time.</li></ol>

<p>If the restrictions are on operation time or if the focus is on first point for improvement, then one can apply compressed time accelerations to reduce the testing time. If the focus is on calendar time (i.e. if there are predefined deadlines), then <a href="Stress_testing" title="Stress testing">intensified stress testing</a> is used.<sup id="cite_ref-Lewis_2-1" class="reference"><a href="#cite_note-Lewis-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-4" class="reference"><a href="#cite_note-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading2"><h2 id="Measurement">Measurement</h2></div>

<p>Software <a href="Availability" title="Availability">availability</a> is measured in terms of <a href="Mean_time_between_failures" title="Mean time between failures">mean time between failures</a> (MTBF).<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup>

</p><p>MTBF consists of <a href="Mean_time_to_failure" class="mw-redirect" title="Mean time to failure">mean time to failure</a> (MTTF) and <a href="Mean_time_to_repair" title="Mean time to repair">mean time to repair</a> (MTTR). MTTF is the difference of time between two consecutive failures and MTTR is the time required to fix the failure.<sup id="cite_ref-6" class="reference"><a href="#cite_note-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup>

</p>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle MTBF=MTTF+MTTR}</annotation>

</semantics>

</math></span><img src="./d98dcfc0d8fe0e0d60bc5ff0d5cbe63ca12d2382.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.505ex; width:28.456ex; height:2.343ex;" alt="{\displaystyle MTBF=MTTF+MTTR}" loading="lazy"></span></dd></dl>

<p>Steady state <a href="Availability" title="Availability">availability</a> represents the percentage the software is operational.

</p>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

</mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A={\frac {MTTF}{MTTF+MTTR}}={\frac {MTTF}{MTBF}}}</annotation>

</semantics>

</math></span><img src="./77ed76eecca121ec75224c1c4efab958c3abffc7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.005ex; width:34.97ex; height:5.343ex;" alt="{\displaystyle A={\frac {MTTF}{MTTF+MTTR}}={\frac {MTTF}{MTBF}}}" loading="lazy"></span></dd></dl>

<p>For example, if MTTF = 1000 hours for a software, then the software should work for 1000 hours of continuous operations.

</p><p>For the same software if the MTTR = 2 hours, then the <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle MTBF=1000+2=1002}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle MTBF=1000+2=1002}</annotation>

</semantics>

</math></span><img src="./d08e77c29ac9d662093a7b7953d985177e7e8295.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.505ex; width:27.083ex; height:2.343ex;" alt="{\displaystyle MTBF=1000+2=1002}" loading="lazy"></span>.

</p><p>Accordingly, <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle A=1000/1002\approx 0.998}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle A=1000/1002\approx 0.998}</annotation>

</semantics>

</math></span><img src="./11d8e6728501de0f9282597e49f0f38e94d042ec.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:23.699ex; height:2.843ex;" alt="{\displaystyle A=1000/1002\approx 0.998}" loading="lazy"></span>

</p><p>Software reliability is measured in terms of failure rate (<span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lambda }</annotation>

</semantics>

</math></span><img src="./b43d0ea3c9c025af1be9128e62a18fa74bedda2a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.355ex; height:2.176ex;" alt="{\displaystyle \lambda }" loading="lazy"></span>).

</p>

<mfrac>

<mrow>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lambda ={\frac {1}{MTBF}}}</annotation>

</semantics>

</math></span><img src="./0f4a26501a71826b41452a5c7216d3e096559013.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:12.873ex; height:5.176ex;" alt="{\displaystyle \lambda ={\frac {1}{MTBF}}}" loading="lazy"></span></dd></dl>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R(t)=e^{-\lambda \cdot t}}</annotation>

</semantics>

</math></span><img src="./a78011e98e141b88b11c75b51173ac8a09b98a0d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:12.115ex; height:3.176ex;" alt="{\displaystyle R(t)=e^{-\lambda \cdot t}}" loading="lazy"></span></dd></dl>

<p>Reliability for software is a number between 0 and 1. Reliability increases when errors or bugs from the program are removed.<sup id="cite_ref-7" class="reference"><a href="#cite_note-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup> There are many software reliability growth models (SRGM) (<a href="List_of_software_reliability_models" title="List of software reliability models">List of software reliability models</a>) including, logarithmic, polynomial, exponential, power, and S-shaped

</p>

<div class="mw-heading mw-heading2"><h2 id="Objectives_of_reliability_testing">Objectives of reliability testing</h2></div>

<p>The main objective of the reliability testing is to test software performance under given conditions without any type of corrective measure using known fixed procedures considering its specifications.

</p>

<div class="mw-heading mw-heading3"><h3 id="Secondary_objectives">Secondary objectives</h3></div>

<p>The secondary objectives of reliability testing is:

</p>

<ol><li>To find perceptual structure of repeating failures.</li>

<li>To find the number of failures occurring in a specified amount of time.</li>

<li>To find the mean life of the software.</li>

<li>To discover the main cause of failure.</li>

<li>Checking the performance of different units of software after taking preventive actions.</li></ol>

<div class="mw-heading mw-heading3"><h3 id="Points_for_defining_objectives">Points for defining objectives</h3></div>

<p>Some restrictions on creating objectives include:

</p>

<ol><li>Behaviour of the software should be defined in given conditions.</li>

<li>The objective should be feasible.</li>

<li>Time constraints should be provided.<sup id="cite_ref-Reliabilityand_8-0" class="reference"><a href="#cite_note-Reliabilityand-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup></li></ol>

<div class="mw-heading mw-heading2"><h2 id="Importance_of_reliability_testing">Importance of reliability testing</h2></div>

<p>The application of computer software has crossed into many different fields, with software being an essential part of industrial, commercial and military systems. Because of its many applications in safety critical systems, software reliability is now an important research area. Although <a href="Software_engineering" title="Software engineering">software engineering</a> is becoming the fastest developing technology of the last century, there is no complete, scientific, quantitative measure to assess them. Software reliability testing is being used as a tool to help assess these software engineering technologies.<sup id="cite_ref-9" class="reference"><a href="#cite_note-9"><span class="cite-bracket">[</span>9<span class="cite-bracket">]</span></a></sup>

</p><p>To improve the performance of software product and software development process, a thorough assessment of reliability is required. Testing software reliability is important because it is of great use for software managers and practitioners.<sup id="cite_ref-10" class="reference"><a href="#cite_note-10"><span class="cite-bracket">[</span>10<span class="cite-bracket">]</span></a></sup>

</p><p>To verify the reliability of the software via testing:

</p>

<ol><li>A sufficient number of test cases should be executed for a sufficient amount of time to get a reasonable estimate of how long the software will execute without failure. Long duration tests are needed to identify defects (such as memory leakage and buffer overflows) that take time to cause a fault or failure to occur.</li>

<li>The distribution of test cases should match the actual or planned operational profile of the software. The more often a function or subset of the software is executed, the greater the percentage of test cases that should be allocated to that function or subset.</li></ol>

<div class="mw-heading mw-heading2"><h2 id="Types_of_reliability_testing">Types of reliability testing</h2></div>

<p>Software reliability testing includes feature testing, load testing, and <a href="Regression_testing" title="Regression testing">regression testing</a>.<sup id="cite_ref-John_11-0" class="reference"><a href="#cite_note-John-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading3"><h3 id="Feature_test">Feature test</h3></div>

<p>Feature testing checks the features provided by the software and is conducted in the following steps:

</p>

<ul><li>Each operation in the software is executed once.</li>

<li>Interaction between the two operations is reduced and</li>

<li>Each operation is checked for its proper execution.</li></ul>

<p>The feature test is followed by the load test.<sup id="cite_ref-John_11-1" class="reference"><a href="#cite_note-John-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup>

</p>

<p>This test is conducted to check the performance of the software under maximum work load. Any software performs better up to some amount of workload, after which the response time of the software starts degrading. For example, a web site can be tested to see how many simultaneous users it can support without performance degradation. This testing mainly helps for <a href="Databases" class="mw-redirect" title="Databases">Databases</a> and <a href="Application_servers" class="mw-redirect" title="Application servers">Application servers</a>. Load testing also requires <a href="Software_performance_testing" title="Software performance testing">software performance testing</a>, which checks how well some software performs under workload.<sup id="cite_ref-John_11-2" class="reference"><a href="#cite_note-John-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading3"><h3 id="Regression_test">Regression test</h3></div>

<p>Regression testing is used to check if any new bugs have been introduced through previous bug fixes. Regression testing is conducted after every change or update in the software features. This testing is periodic, depending on the length and features of the software.<sup id="cite_ref-John_11-3" class="reference"><a href="#cite_note-John-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup>

</p>

<div class="mw-heading mw-heading2"><h2 id="Test_planning">Test planning</h2></div>

<p>Reliability testing is more costly compared to other types of testing. Thus while doing reliability testing, proper management and planning is required. This plan includes testing process to be implemented, data about its environment, test schedule, test points, etc.

</p>

<div class="mw-heading mw-heading3"><h3 id="Problems_in_designing_test_cases">Problems in designing test cases</h3></div>

<p>Some common problems that occur when designing test cases include:

</p>

<ul><li>Test cases can be designed simply by selecting only valid input values for each field in the software. When changes are made in a particular module, the previous values may not actually test the new features introduced after the older version of software.</li>

<li>There may be some critical runs in the software which are not handled by any existing test case. Therefore, it is necessary to ensure that all possible types of test cases are considered through careful test case selection.<sup id="cite_ref-John_11-4" class="reference"><a href="#cite_note-John-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li></ul>

<div class="mw-heading mw-heading2"><h2 id="Reliability_enhancement_through_testing">Reliability enhancement through testing</h2></div>

<p>Studies during development and design of software help for improving the reliability of a product. Reliability testing is essentially performed to eliminate the failure mode of the software. Life testing of the product should always be done after the design part is finished or at least the complete design is finalized.<sup id="cite_ref-Liwis_12-0" class="reference"><a href="#cite_note-Liwis-12"><span class="cite-bracket">[</span>12<span class="cite-bracket">]</span></a></sup>

Failure analysis and design improvement is achieved through testings.

</p>

<div class="mw-heading mw-heading3"><h3 id="Reliability_growth_testing">Reliability growth testing</h3></div>

This testing is used to check new prototypes of the software which are initially supposed to fail frequently. The causes of failure are detected and actions are taken to reduce defects.

Suppose T is total accumulated time for prototype. n(T) is number of failure from start to time T. The graph drawn for n(T)/T is a straight line. This graph is called Duane Plot. One can get how much reliability can be gained after all other cycles of test and fix it.

</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\begin{alignedat}{5}ln\left[{\frac {n\left(T\right)}{T}}\right]=-\alpha ln\left(T\right)+b;\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ .....Eq:1\end{alignedat}}}">

<mtr>

<mtd>

<mrow>

<mfrac>

<mrow>

<mrow>

</mrow>

</mfrac>

</mrow>

</mrow>

<mrow>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{alignedat}{5}ln\left[{\frac {n\left(T\right)}{T}}\right]=-\alpha ln\left(T\right)+b;\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ .....Eq:1\end{alignedat}}}</annotation>

</semantics>

</math></span><img src="./a6727368bb16b7d90c1aac047bccb625ddd15648.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:48.756ex; height:6.176ex;" alt="{\displaystyle {\begin{alignedat}{5}ln\left[{\frac {n\left(T\right)}{T}}\right]=-\alpha ln\left(T\right)+b;\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ .....Eq:1\end{alignedat}}}" loading="lazy"></span>

</p><p>solving eq.1 for n(T),

</p><p><span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\begin{alignedat}{5}n\left(T\right)=KT^{1-\alpha };\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ......Eq:2\end{alignedat}}}">

<mtr>

<mtd>

<mrow>

</mrow>

<msup>

</mrow>

</msup>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{alignedat}{5}n\left(T\right)=KT^{1-\alpha };\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ......Eq:2\end{alignedat}}}</annotation>

</semantics>

</math></span><img src="./d16725fe7c40c6d3b962d0a32495bf18acee8173.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:40.463ex; height:3.176ex;" alt="{\displaystyle {\begin{alignedat}{5}n\left(T\right)=KT^{1-\alpha };\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ......Eq:2\end{alignedat}}}" loading="lazy"></span>

</p><p>where K is e^b.

If the value of alpha in the equation is zero the reliability can not be improved as expected for given number of failure. For alpha greater than zero, cumulative time T increases. This explains that number of the failures doesn't depends on test lengths.

</p>

<div class="mw-heading mw-heading3"><h3 id="Designing_test_cases_for_current_release">Designing test cases for current release</h3></div>

<p>If new features are being added to the current version of software, then writing a test case for that operation is done differently.

</p>

<ul><li>First plan how many new test cases are to be written for current version.</li>

<li>If the new feature is part of any existing feature, then share the test cases of new and existing features among them.</li>

<li>Finally combine all test cases from current version and previous one and record all the results.<sup id="cite_ref-John_11-5" class="reference"><a href="#cite_note-John-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li></ul>

<p>There is a predefined rule to calculate count of new test cases for the software. If N is the probability of occurrence of new operations for new release of the software, R is the probability of occurrence of used operations in the current release and T is the number of all previously used test cases then

</p>

<mtr>

<mtd>

<msub>

</mrow>

</msub>

<mrow>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{alignedat}{5}NewTestcases_{(currentrelease)}=\left({\frac {N}{R}}\right)*T\end{alignedat}}}</annotation>

</semantics>

</math></span><img src="./f48b2adc2de6cf553643fdd75a666633d98b699f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.505ex; width:41.128ex; height:6.176ex;" alt="{\displaystyle {\begin{alignedat}{5}NewTestcases_{(currentrelease)}=\left({\frac {N}{R}}\right)*T\end{alignedat}}}" loading="lazy"></span>

</pre>

<div class="mw-heading mw-heading2"><h2 id="Reliability_evaluation_based_on_operational_testing">Reliability evaluation based on operational testing</h2></div>

<p>The method of operational testing is used to test the reliability of software. Here one checks how the software works in its relevant operational environment. The main problem with this type of evaluation is constructing such an operational environment. Such type of simulation is observed in some industries like nuclear industries, in aircraft, etc. Predicting future reliability is a part of reliability evaluation.

</p><p>There are two techniques used for operational testing to test the reliability of software:

</p>

<dl><dt>Steady state reliability estimation</dt>

<dd>In this case, we use feedback from delivered software products. Depending on those results, we can predict the future reliability for the next version of product. This is similar to sample testing for physical products.</dd>

<dt>Reliability growth based prediction</dt>

<dd>This method uses documentation of the testing procedure. For example, consider a developed software and that we are creating different new versions of that software. We consider data on the testing of each version and based on the observed trend, we predict the reliability of the new version of software.<sup id="cite_ref-bare_url_13-0" class="reference"><a href="#cite_note-bare_url-13"><span class="cite-bracket">[</span>13<span class="cite-bracket">]</span></a></sup></dd></dl>

<div class="mw-heading mw-heading3"><h3 id="Reliability_growth_assessment_and_prediction">Reliability growth assessment and prediction</h3></div>

<p>In the assessment and prediction of software reliability, we use the reliability growth model. During operation of the software, any data about its failure is stored in statistical form and is given as input to the reliability growth model. Using this data, the reliability growth model can evaluate the reliability of software.

</p><p>Much data about reliability growth model is available with probability models claiming to represent failure process. But there is no model which is best suited for all conditions. Therefore, we must choose a model based on the appropriate conditions.

</p>

<div class="mw-heading mw-heading3"><h3 id="Reliability_estimation_based_on_failure-free_working">Reliability estimation based on failure-free working</h3></div>

<p>In this case, the reliability of the software is estimated with assumptions like the following:

</p>

<ul><li>If a defect is found, then is it going to be fixed by someone.</li>

<li>Fixing the defect will not have any effect on the reliability of the software.</li>

<li>Each fix in the software is accurate.<sup id="cite_ref-bare_url_13-1" class="reference"><a href="#cite_note-bare_url-13"><span class="cite-bracket">[</span>13<span class="cite-bracket">]</span></a></sup></li></ul>

<ul><li><a href="Software_testing" title="Software testing">Software testing</a></li>

<li><a href="Load_testing" title="Load testing">Load testing</a></li>

<li><a href="Regression_testing" title="Regression testing">Regression testing</a></li>

<li><a href="Reliability_engineering" title="Reliability engineering">Reliability engineering</a></li>

<li><a href="List_of_software_reliability_models" title="List of software reliability models">List of software reliability models</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite class="citation book cs1"><i>Software Reliability</i>. Hoang Pham.</cite></span>

</li>

<li id="cite_note-Lewis-2"><span class="mw-cite-backlink">^ <a href="#cite_ref-Lewis_2-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-Lewis_2-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFE.E.Lewis1988" class="citation journal cs1">E.E.Lewis (1988). "Introduction to Reliability Engineering". <i>Nuclear Technology</i>. <b>82</b> (2): 235. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/1988NucTe..82..235K">1988NucTe..82..235K</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.13182%2FNT88-A34110">10.13182/NT88-A34110</a>.</cite></span>

</li>

</li>

<li id="cite_note-4"><span class="mw-cite-backlink"><b><a href="#cite_ref-4">^</a></b></span> <span class="reference-text"><cite class="citation cs2"><i>IEEE Recommended Practice on Software Reliability</i>, IEEE, <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Fieeestd.2017.7827907">10.1109/ieeestd.2017.7827907</a>, <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-5044-3648-9</bdi></cite></span>

</li>

<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text"><cite id="CITEREFRoger_Pressman1982" class="citation book cs1">Roger Pressman (1982). <span class="id-lock-registration" title="Free registration required"><a rel="nofollow" class="external text" href="https://archive.org/details/softwareengineeprer00pres"><i>Software Engineering A Practitioner's Approach</i></a></span>. McGraw Hill. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-07-050781-4</bdi>.</cite></span>

</li>

<li id="cite_note-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-6">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.softwaretestinggenius.com/articalDetails.php?qry=963">"Approaches to Reliability Testing & Setting of Reliability Test Objectives"</a>.</cite></span>

</li>

<li id="cite_note-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-7">^</a></b></span> <span class="reference-text"><cite id="CITEREFAditya_P._Mathur" class="citation book cs1">Aditya P. Mathur. <i>Foundations of Software Testing</i>. Pearson publications.</cite></span>

</li>

<li id="cite_note-Reliabilityand-8"><span class="mw-cite-backlink"><b><a href="#cite_ref-Reliabilityand_8-0">^</a></b></span> <span class="reference-text"><cite class="citation book cs1"><i>Reliability and life testing handbook</i>. Dimitri kececioglu.</cite></span>

</li>

<li id="cite_note-9"><span class="mw-cite-backlink"><b><a href="#cite_ref-9">^</a></b></span> <span class="reference-text"><cite class="citation book cs1"><i>A Statistical Basis for Software Reliability Assessment</i>. M. xie.</cite></span>

</li>

<li id="cite_note-10"><span class="mw-cite-backlink"><b><a href="#cite_ref-10">^</a></b></span> <span class="reference-text"><cite class="citation book cs1"><i>Software Reliability modelling</i>. M. Xie.</cite></span>

</li>

<li id="cite_note-John-11"><span class="mw-cite-backlink">^ <a href="#cite_ref-John_11-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-John_11-1"><sup><i><b>b</b></i></sup></a> <a href="#cite_ref-John_11-2"><sup><i><b>c</b></i></sup></a> <a href="#cite_ref-John_11-3"><sup><i><b>d</b></i></sup></a> <a href="#cite_ref-John_11-4"><sup><i><b>e</b></i></sup></a> <a href="#cite_ref-John_11-5"><sup><i><b>f</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFJohn_D._Musa2004" class="citation book cs1">John D. Musa (2004). <i>Software reliability engineering: more reliable software, faster and cheaper</i>. McGraw-Hill. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-07-060319-7</bdi>.</cite></span>

</li>

<li id="cite_note-Liwis-12"><span class="mw-cite-backlink">^ <a href="#cite_ref-Liwis_12-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-Liwis_12-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFE.E.Liwis1995" class="citation book cs1">E.E.Liwis (1995-11-15). <i>Introduction to Reliability Engineering</i>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-471-01833-3</bdi>.</cite></span>

</li>

<li id="cite_note-bare_url-13"><span class="mw-cite-backlink">^ <a href="#cite_ref-bare_url_13-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-bare_url_13-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite class="citation web cs1">"Problem of Assessing reliability". <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.9831">10.1.1.104.9831</a></span>.</cite> <span class="cs1-visible-error citation-comment"><code class="cs1-code">{{cite web}}</code>: </span><span class="cs1-visible-error citation-comment">Missing or empty <code class="cs1-code">|url=</code> (help)</span></span>

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>

<ul><li><a rel="nofollow" class="external text" href="http://www.weibull.com/hotwire/issue94/relbasics94.htm/">Mean Time Between Failure</a></li>

<li><a rel="nofollow" class="external text" href="http://www.weibull.com/basics/accelerated.htm/">Software Life Testing</a></li></ul>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1236075235">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style></div><div role="navigation" class="navbox" aria-labelledby="Software_testing107" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><style data-mw-deduplicate="TemplateStyles:r1239400231">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div id="Software_testing107" style="font-size:114%;margin:0 4em"><a href="Software_testing" title="Software testing">Software testing</a></div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">Test levels</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Acceptance_testing" title="Acceptance testing">Acceptance testing</a></li>

<li><a href="System_integration_testing" title="System integration testing">System integration testing</a></li>

<li><a href="System_testing" title="System testing">System testing</a></li>

<li><a href="Integration_testing" title="Integration testing">Integration testing</a></li>

<li><a href="Unit_testing" title="Unit testing">Unit testing</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Test types,<br>techniques,<br><a href="Software_testing_tactics" title="Software testing tactics">tactics</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><td colspan="2" class="navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="A/B_testing" title="A/B testing">A/B testing</a></li>

<li><a href="Benchmark_(computing)" title="Benchmark (computing)">Benchmark</a></li>

<li><a href="Compatibility_testing" title="Compatibility testing">Compatibility testing</a></li>

<li><a href="Concolic_testing" title="Concolic testing">Concolic testing</a></li>

<li><a href="Concurrent_testing" title="Concurrent testing">Concurrent testing</a></li>

<li><a href="Conformance_testing" title="Conformance testing">Conformance testing</a></li>

<li><a href="Continuous_testing" title="Continuous testing">Continuous testing</a></li>

<li><a href="Destructive_testing" title="Destructive testing">Destructive testing</a></li>

<li><a href="Development_testing" title="Development testing">Development testing</a></li>

<li><a href="Differential_testing" title="Differential testing">Differential testing</a></li>

<li><a href="Dynamic_program_analysis" title="Dynamic program analysis">Dynamic program analysis</a></li>

<li><a href="Installation_testing" title="Installation testing">Installation testing</a></li>

<li><a href="Negative_testing" title="Negative testing">Negative testing</a></li>

<li><a href="Random_testing" title="Random testing">Random testing</a></li>

<li><a href="Regression_testing" title="Regression testing">Regression testing</a></li>

<li><a href="Security_testing" title="Security testing">Security testing</a></li>

<li><a href="Smoke_testing_(software)" title="Smoke testing (software)">Smoke testing (software)</a></li>

<li><a href="Software_performance_testing" title="Software performance testing">Software performance testing</a></li>

<li><a href="Stress_testing_(software)" title="Stress testing (software)">Stress testing</a></li>

<li><a href="Symbolic_execution" title="Symbolic execution">Symbolic execution</a></li>

<li><a href="Test_automation" title="Test automation">Test automation</a></li>

<li><a href="Usability_testing" title="Usability testing">Usability testing</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">[x]-box<br>style</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Black-box_testing" title="Black-box testing">Black-box testing</a>

<ul><li><a href="All-pairs_testing" title="All-pairs testing">All-pairs testing</a></li>

<li><a href="Exploratory_testing" title="Exploratory testing">Exploratory testing</a></li>

<li><a href="Fuzz_testing" class="mw-redirect" title="Fuzz testing">Fuzz testing</a></li>

<li><a href="Model-based_testing" title="Model-based testing">Model-based testing</a></li>

<li><a href="Scenario_testing" title="Scenario testing">Scenario testing</a></li></ul></li>

<li><a href="Grey-box_testing" class="mw-redirect" title="Grey-box testing">Grey-box testing</a></li>

<li><a href="White-box_testing" title="White-box testing">White-box testing</a>

<ul><li><a href="API_testing" title="API testing">API testing</a></li>

<li><a href="Mutation_testing" title="Mutation testing">Mutation testing</a></li>

<li><a href="Static_testing" class="mw-redirect" title="Static testing">Static testing</a></li></ul></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">See also</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Graphical_user_interface_testing" title="Graphical user interface testing">Graphical user interface testing</a></li>

<li><a href="Manual_testing" title="Manual testing">Manual testing</a></li>

<li><a href="Orthogonal_array_testing" title="Orthogonal array testing">Orthogonal array testing</a></li>

<li><a href="Pair_testing" title="Pair testing">Pair testing</a></li>

<li><a href="Soak_testing" title="Soak testing">Soak testing</a></li>

<li><a href="Stress_testing" title="Stress testing">Stress testing</a></li>

<li><a href="Web_testing" title="Web testing">Web testing</a></li></ul>

</div></td></tr></tbody></table></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-01-26" href="https://en.wikipedia.org/wiki/?title=Software_reliability_testing&oldid=1272005128">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>